Generalized Momentum-Based Methods: A Hamiltonian Perspective

We take a Hamiltonian-based perspective to generalize Nesterov's accelerated gradient descent and Polyak's heavy ball method broad class of momentum methods in the setting (possibly) constrained minimization Euclidean non-Euclidean normed vector spaces. Our leads generic unifying nonasymptotic analysis convergence these both function value (in convex optimization) norm unconstrained, possibly nonconvex, optimization). approach relies upon time-varying Hamiltonian that produces generalized as its equations motion. The for is intuitive based on conserved quantities time-dependent Hamiltonian.